Data and Task Alignment in Distributed Memory Architectures

Diierent alignments of multi-dimensional arrays on a mesh-connected SIMD architecture result in diierent communication patterns during parallel program execution. In this paper a compile-time selection of data alignment that minimizes communication cost is discussed. First, it is shown that the selection is computationally as hard as a sub-class of the well-known problem of nding the closest vector in a lattice. The NP-hardness of the latter problem is proven. Then, two algorithms for exact minimum solution are discussed. Although the complexity of these algorithms is exponential, for small lattices often generated by parallel scientiic computation the execution times of these algorithms may be acceptable. A polynomial-time algorithm for nding an approximate solution is also described. Finally, improvements in communication cost resulting from alignments for randomly generated graphs are presented.